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ABSTRACT 

Each report in "The State of Chicago Public Schools: 1993 to 
2000" makes up part of a comprehensive picture of how high schools changed 
under Phase 2 of the Chicago, Illinois, educational reform efforts. This 
report looks at student performance. Data from a variety of sources show that 
in recent years the high school eligible students in Chicago's public schools 
improved on a number of measures. These trends exist even though the analyses 
include students who dropped out of school between eighth and ninth grade or 
were sent to Academic Preparatory Centers. More students were on track their 
first year after elementary school (received no more than one failing grade 
in a core course and had enough credits to assume sophomore status on time, 
passed the algebra/geometry sequence by the end of their second year, and 
passed an honors class sometime in their first year. Somewhat more students 
also completed a college preparatory program and passed an honors or advanced 
placement course over 4 years. The percent of students graduating by age 18 
rose slightly. Finally, the average scores on the Tests of Achievement and 
Proficiency (TAP). rose substantially for the subset of students who enrolled 
in the ninth grade. Overall, this is good news for the Chicago public high 
schools. However, this good news is qualified by’ the fact that student- 
performance itself, even by 2000, was still very poor on most measures. Fewer 
than half of all students graduated, barely half were on track after their 
freshman year, and the dropout rate remained above 40%. Policies aimed at 
bringing in better prepared students appear to have worked, and these account 
for much of the improvement in student performance . No particular type of 
school was especially effective in improving students' performance on the 
TAP. Four appendixes contain sample data, technical notes, a description of 
first-year course taking, and a chart of change in students taking the TAP by 
performance on the eighth grade Iowa Tests of Basic Skills. (Contains 4 
tables, 29 figures, and 23 references.) ( SLD) 
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THE WAVE OF REFORMS THAT FIRST HIT THE CHICAGO 
Public Schools in the late 1980s focused primarily on greater local con- 
trol over schools. During this period (Phase I), there was considerable 
improvement in Chicago public elementary schools. Change in the high 
schools, however, was minimal. 



Beginning in 1995, a second wave of reform 
(Phase II) focused on strong mayoral control, high 
stakes accountability, and increased attention from 
system administrators. Phase II included the 
systemwide redesign of high schools. 

This series tracks the performance of Chicago 
public high school students from 1993 to 2000 
using data from the last two years of Phase I 
reform as a baseline for looking at the first five 
years of Phase II. Each report in The State of 
Chicago Public Schools : 1993 to 2000 series makes 
up part of a comprehensive picture of how high 
schools changed under Phase II. This report looks 
at student performance. Declining High School 
Enrollment: An Exploration of Causes tracks 
changes in high school enrollment and explores 
possible causes. Changing Special Education 
Enrollments: Causes and Distribution among 
Schools , examines the increase in the percent of 
students eligible for special education services and 
their distribution across schools. 

Student Performance 

Student Performance: Course Taking, Test Scores, and 
Outcomes shows that in recent years high school 
eligible students in Chicago's public schools 
improved on a number of measures. These trends 
exist even though we include in our analyses the 



students who dropped out of school between 
eighth and ninth grade, or were sent to Academic 
Preparatory Centers. More students were on track 
their first year after elementary school (received no 
more than one failing grade in a core course and 
had enough credits to assume sophomore status 
on time), passed the algebra/ geometry sequence by 
the end of their second year, and passed an honors 
class sometime in their first year. Somewhat more 
students also completed a college preparatory 
program and passed an honors or advanced 
placement course over four years. The percent of 
students graduating by age 18 rose slightly, and 
the percent of students dropping out by age 18 fell 
slightly. Finally, the average score on the Tests of 
Achievement and Proficiency (TAP) rose substan- 
tially for the subset of students who enrolled in 
the ninth grade. 

Overall, this is good news for Chicago public 
high schools — students' likelihood of succeeding 
in school has increased. At the same time, this 
“good news" is qualified by the fact that student 
performance itself, even by 2000, was still very 
poor on most measures. Fewer than half of all 
students graduated, barely half were on track after 
their freshman year, and the dropout rate re- 
mained above 40 percent. 
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When measuring the effects of high school 
reform policies, the news is mixed. Policies aimed 
at bringing in better-prepared students appear to 
have worked well and account for much of the 
improvement in student performance. The 
systems redesign of high schools in 1997, how- 
ever, appears to have had only a modest impact on 
increasing the rate of the development of students’ 
skills. The rates at which students graduated, 
stayed on track, passed the algebra/geometry 
sequence, and completed a college preparatory 
program appear to be, in part, due to something 
besides better-prepared students. On the other 
hand, the increase in the frequency with which 
students took honors and advanced placement 
courses seems to have occurred primarily as a 
result of students leaving elementary school better 
prepared for high school. 

No particular type of school (neighborhood, 
vocational, charter or small, extended elementary, 
or selective admissions) was especially effective at 
improving students’ performance on the TAP; 
individual schools from each category stood out. 
When looking at dropout rates, however, there is a 
noticeable pattern. Charter schools had substan- 
tially lower dropout rates after controlling for the 
characteristics of their incoming students. 
Selective admissions schools also did well. 

While Academic Preparatory Centers (APCs) 
generally had mixed results, a few had more 
success than expected in keeping their students 
from dropping out. 

OAer Reports in the Series 

Declining High School Enrollment 

Declining High School Enrollment: An Exploration 
of Causes documents changes in high school 
enrollment from 1993 to 2000 and examines why 
they occurred. Analysis shows that the introduc- 
tion of the promotion gate policy to CPS elemen- 
tary schools in the 1993-96 school year had a 
profound effect on high school enrollment. As 



lower achieving eighth-grade students were re- 
tained or sent to APCs, the size of ninth-grade 
cohorts shrank. Successive grades were affected as 
smaller cohorts moved through high school. The 
better-prepared students who did make it to grade 
nine were less likely to spend more than four years 
in high school than in the past, thereby further 
depressing enrollment. As a result of this reduc- 
tion in high school course repetition, the increase 
in eighth-grade retention was not accompanied by 
a decline in graduation rates by age 18. 

Enrollment and Distribution of 
Special Education Students 

Changing Special Education Enrollments: Causes 
and Distribution among Schools examines in detail 
the upward trend of special education enrollment 
in high schools. The enrollment of students with 
disabilities in ninth grade increased substantially 
over the period of our study, from 11.5 percent in 
the 1993-94 school year to 16.4 percent in 1999- 
GO. During this period, a larger percentage of 
elementary students were identified as having 
learning disabilities, particularly in the later 
grades. Although the proportion of students with 
disabilities that was retained only rose slightly, the 
proportion of general education students that was 
retained or sent to an APC rose dramatically. 
Moreover, students with disabilities were heavily 
concentrated in neighborhood high schools (rather 
than selective admissions, charter, small, or ex- 
tended elementary schools). Eleven neighborhood 
high schools, all on probation and located in areas 
with disproportionately high levels of low-income 
residents, experienced an especially large increase 
in their enrollment of students with special 
needs — from 16.3 percent in the 1993-94 school 
year, to 30.1 percent in 1999—00. The overall 
growth of special education enrollment across the 
system resulted in the increased separation of 
students with learning disabilities from general 
education students in high school classrooms, 
especially in schools where a larger proportion of 
students with disabilities was served. 
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The Unexpected Consequence of Reform Policies 

Looking across the reports, we see that some of the 
most dramatic changes in CPS high schools 
between 1993 and 2000 were the consequences of 
changes in CPS elementary schools. This is 
especially pronounced when looking at the long- 
term impact of the eighth-grade promotion gate 
adopted by the system in 1995-96. 

One of the anticipated consequences of the 
policy to end social promotion was a student 
population better prepared to cope with the 
demands of high school. In Student Performance, 
trends in graduation, course-taking, and test scores 
all show improvement, much of which is attribut- 
able to better-prepared students in the high 
schools. On the other hand, Declining High School 
Enrollment shows that high school enrollment 
declined substantially between 1993 and 2000 due 
in part to fewer students passing the promotion 
gate and, because those who did enroll were better 
prepared and less likely to repeat a grade, students 
moved through the system more quickly. In 
Changing Special Education Enrollments , we see 
that although the general education students who 
enrolled in high school were better prepared, they 
comprised a smaller percentage of incoming ninth 
graders. Because relatively fewer general education 



students were being promoted, high schools 
enrolled a greater concentration of students with 
disabilities. This is especially true for those high 
schools whose traditional enrollment was made up 
of mostly low achieving students who were not 
passing the promotion gate. The higher concentra- 
tion of students with disabilities made their 
inclusion in general education classrooms more 
complex and difficult. 

The reverberating effects of the eighth-grade 
promotion gate show that before the system 
adopts a policy to prompt change in the elemen- 
tary schools, it should be considered in light of its 
possible long-term effects on high schools. 

New Information on CPS High Schools 

Outcomes for the 2000-01 school year show no 
substantial difference from the trends presented in 
these reports, with the exception of the distribu- 
tion of students with disabilities being signifi- 
cantly less focused on neighborhood high schools. 
Updated data for 2000-01 will be posted on the 
Consortiums website (www.consortium- 
chicago.org). A planned fourth report in this 
series will look more specifically at the climate 
inside high schools. The projected date for this 
reports publication is January 2003. 
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OVER THE COURSE OF THE 1990s, CHICAGO PUBLIC 
schools were the subject of a great deal of public attention, broadly scaled 
reforms, and sweeping changes. Under the decentralization reforms of 
the first half of the decade, a significant number of elementary schools 
made well-documented progress in improving school organization, in- 
struction, and student performance. 



High schools, on the other hand, showed few 
signs of improvement during this period. In 
Charting Reform in Chicago: The Students Speak 
(1996), Consortium researchers reported that 
most high schools were “designed for failure” and 
that few students were prepared to participate in 
the labor market or enter higher education. Over 
the second half of the decade, the Chicago Public 
Schools (CPS) tried to spur change in the high 
schools by implementing targeted reforms and 
systemwide redesign. 1 Other policy changes, 
especially the ending of social promotion in 
elementary schools, had major implications for 
high schools as well. 

Although other researchers have carefully 
documented and evaluated specific reform strate- 
gies (Hess and Cytrynbaum, forthcoming), this 
report looks at broad indicators of improvement 
in high school student performance from 1993 to 
2000. We address three issues: What improve- 
ment, if any, has there been in student perfor- 
mance? What is the absolute level of performance? 
Is improvement the result of real changes in the 
high schools, or only a consequence of having 
better-prepared students? We analyze traditional 
performance indicators such as standardized test 
scores and graduation and dropout rates, and new 
indicators such as measures of whether students 



are “on track” to graduate one year after elemen- 
tary school and whether they complete a college 
preparatory program or pass any honors courses 
(see Table 1). In addition, we also look at how 
standardized test scores and two-year dropout rates 
compare among the different types of CPS high 
schools. (See Appendix I for descriptions of our 
sample group for each analysis.) 




CPS high school redesign centered on four dis- 
tinct sets of policies. Although these were imple- 
mented in individual schools to widely varying 
degrees, their major provisions were the same 
across the system: 

• Improve the quality of students entering high 
school. 

0 Mandate more challenging work. 

• Help students meet the demands of higher 
standards. 

® Focus resources and penalties on extremely 
low performing schools. 
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Table 1: Outcomes Included in this Study 





Length of Time Measured 


Outcome Type 


One and Two Year 


Four Year 


Basic Measures 

(adequate 

performance) 


On track 


Graduation 

Dropout 


Course-Taking 

(moderate 

performance) 


Algebra-geometry 

sequence 


Full college 
sequence 

College sequence 
without foreign 
language 


Advanced Course- 
Taking 

(advanced 

performance) 


At least one honors 
course in the 
first year 


At least one 
honors course 

At least one 
AP course 


Achievement 

Scores 


TAP reading 
TAP math 





Better Prepared Students 

A strong push to improve the quality of students 
entering high school entailed a systemwide policy 
requiring students to meet specified performance 
levels before they could matriculate. This directly 
addressed a claim long made by CPS high school 
teachers that they would have more success in the 
classroom if students came to high school better 
prepared. 

Reform efforts in the elementary schools 
resulted in several years of rising eighth-grade 
scores on the Iowa Tests of Basic Skills (ITBS). 2 
Eighth graders who did not meet a specified ITBS 
cut-off score were generally retained in elementary 
school instead of promoted to ninth grade, or if a 
student would be turning 15 by December 1 of 
the following academic year, he or she was sent to 
an Academic Preparatory Center (APC), ideally 
until the ITBS cut-off score was met. APCs were 
designed to help students with weak skills make 
the transition from elementary to high school by 
providing additional academic and social support. 
With the implementation of the ITBS promotion 
gate and the creation of APCs, Chicago public 
high schools enrolled fewer students with weak 
basic skills and, as a result, classrooms were filled 
with higher achieving students. 



In addition to the above initiatives, CPS 
worked to attract and retain more able students. 

In order to achieve this, the system planned a 
college preparatory selective admissions high 
school for each of the city’s six regions (at the time 
of publication, all but one was open). These 
schools were quite successful in encouraging 
better-prepared students to remain at, or enroll in, 
CPS high schools. 3 The system also developed a 
wide range of magnet programs inside existing 
neighborhood high schools to attract students who 
might have otherwise enrolled in a selective 
admissions school or left the system altogether. 

A More Rigorous Curriculum 

Research suggests that without stiff requirements, 
students often shy away from taking a heavy 
course load or more challenging classes. And yet, 
when they are required to take these classes, their 
standardized test scores rise and their dropout 
rates decrease. 4 The impact of a tougher academic 
program on minority students is even greater — 
they earn higher wages after high school, and the 
racial divide in standardized test scores and gradu- 
ation rates diminishes. 5 

Starting with the freshman class of 1997-98, 
CPS students are required to complete a rigorous 
program of study that meets the entrance require- 
ments of competitive colleges like the University 
of Illinois at Urbana-Champaign: four years of 
English, three years of math (algebra, geometry, 
and advanced algebra/ trigonometry), three years 
of social science, and two years of foreign lan- 
guage. Non-academic classes such as fine arts, 
physical education, and career exploration are also 
required. 6 These new graduation requirements are 
comparable to the “New Basics” recommended in 
A Nation at Risk (1983). In fact, with its foreign 
language requirement, CPS’s mandated curricu- 
lum actually exceeds the New Basics. 

Increased Student Support 

To counter a possible increase in the number of 
dropouts brought about by the more challenging 
curriculum, CPS introduced a range of initiatives 
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to help students through high school. Some 
schools were reorganized into extended elemen-. 
tary schools so students could complete ninth 
grade in a smaller, more familiar environment. 
Junior and senior academies were set up within 
high schools to smooth the transition from 
elementary school for ninth graders and estab- 
lish more intimate and personable learning 
communities. Ail schools were required to 
launch an advisory program to provide students 
with a teacher-advisor to whom they could turn 
for advice and guidance. Recovery programs 
were implemented to help students who failed a 
course in the fall semester pass in the spring, 
and evening high school programs were opened 
for students to make up credits needed for 
graduation. Each schools commitment to these 
initiatives, and the extent of their implementa- 
tion, varied greatly. 7 

Attention for Low Performing Schools 

CPS imposed a combination of penalties and 
increased resources on extremely low performing 
high schools through the processes of remediation, 
probation, reconstitution, reengineering, and 
intervention. For the period of this study, high 



schools that were performing poorly could be 
placed under remediation and required to submit 
annual school improvement plans to the CPS 
Central Office. Schools in which less than 15 
percent of students met national norms on the 
Tests of Achievement and Proficiency (TAP) faced 
probation. Schools on probation were monitored 
until test scores exceeded the threshold or they 
showed significant improvement in five essential 
areas: school leadership, professional development 
and training, instructional program, learning 
environment, and parent and community rela- 
tions. 8 Probation schools received approximately 
$100,000 in additional funds and were assigned 
an external partner from the educational commu- 
nity. In some cases, a business manager was 
appointed to help manage day-to-day affairs so 
that the principal could focus on other tasks. 

Schools failing to make adequate progress under 
probation faced reconstitution, reengineering, or 
intervention. Under these designations, all teach- 
ers, principals, and Local School Council mem- 
bers at these schools were placed under the threat 
of dismissal. In addition, the schools were assigned 
external partners and received $500,000 in addi- 
tional funding. 
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UNDERTHE SECOND PHASE OF REFORM IN THE LATE 1990 s, 
the Chicago Public Schools mounted an aggressive campaign to improve 
its high schools. This campaign had four goals: improve the quality of 
entering ninth-grade students; increase the rigor of the academic pro- 
gram; provide more support for students; and focus increased attention 
on extremely low performing schools. 



The first of these, improving the quality of 
incoming students, was greatly affected by the 
systems implementation of the policy to end social 
promotion that took effect in the 1995—96 school 
year. Under this policy, eighth graders who did not 
meet a specified cut-off score on the ITBS typi- 
cally were retained or sent to an APC. 

Obviously, this policy had a substantial impact 
on the characteristics of students entering high 
school and has made more difficult the problem of 
measuring accurately the impact of the systems 
other high school reforms, especially in a study of 
trends over time. Once the weakest students are 
removed, the performance of those who do go on 
to high school improves by default — higher 
achieving students are the only ones being mea- 
sured. To address this, in this study we distinguish 
between two groups of students: those who enroll 
in ninth grade and all students who leave elemen- 
tary school (or all “high school eligible” students). 
This second group includes students who enroll in 
ninth grade, APCs, or drop out between eighth 
and ninth grade. 

We include both groups — all high school 
eligible students— in our measures of improve- 
ment (neither APC nor eighth-grade students take 
the TAP so we can not include them in our 



calculations of scores). In doing so, we ensure that 
we track a similar population over time and are 
better able to determine whether improvements 
are the result of better-prepared entering students 
or of changes in the high schools. To adjust for 
changes in incoming students, we control for race/ 
ethnicity, gender, socio-economic status, special 
education classification, age, if a student took the 
ITBS, if a student was returning to CPS from 
outside the system or entering the system anew, 
and eighth-grade ITBS scores. 9 These are our 
adjusted outcomes. (See Appendix II for a discus- 
sion of how we adjusted ITBS and TAP scores to 
calculate accurate means.) 

Rising ITBS Scores and Declining Leave Bates: 
Improving the Pool of Entering Students 

Of all the characteristics we control for, the only 
one with substantial change was student prepara- 
tion as measured by eighth-grade ITBS scores. 
With the weakest students being retained, enroll- 
ing in APCs, or dropping out, one could assume 
that students entering ninth grade after the imple- 
mentation of the promotion gate had higher 
scores. Indeed, ITBS reading scores for entering 
ninth graders rose from an average of 7.7 grade 



10 



THE STATE OF CHICAGO PUBLIC HIGH SCHOOLS: 1993 to 2000 



equivalents (GEs) in 1993-94 to 8.4 in 1999-00 
(see Figure l). 10 In math, scores rose from 7.6 to 
8.7 (see Figure 2). For those students who left 
eighth grade but did not enroll in high school 
(i.e., went to an APC or dropped out), scores 
declined sharply after the policy’s adoption. Math 
scores recovered the following year, but reading 
scores only regained their 1993-94 levels in the 
last year of our study. (The average scores for all 
high school eligible students between 1993 and 
2000 were the same as scores for the ninth-grade 
only population because the number of entering 
ninth graders far exceeded the number of APC 
students and eighth-grade dropouts.) On average, 
CPS students were leaving elementary school 
better prepared. 



There was also an increase in the percent of 
high achieving students going on to CPS high 
schools. In general, high achieving students were 
still more likely than others to leave the system 
after elementary school. However, between the 
1993-96 and 1999-00 school years, the propor- 
tion of high achieving students who transferred 
out decreased by 30 percent, from 21.1 to 1 4. 8 
(see Figure 3). 11 By the fall of 1999, the leave rate 
among students in the top quartile was only 
slightly higher than that of students in the other 
three quartiles. This decline may be attributed in 
some part to the opening of several new selective 
admissions schools. 12 It is also important to note 
that leave rates for students in the bottom two 
quartiles increased slightly during this same 
period. 



Figure 1 

ITBS Reading Scores Rise for Entering Ninth Graders, 
Not for APC Students or Pre-Ninth-Grade Dropouts 

Last year of elementary school 





, , Ninth-Grade f ( APC Students and 

Students Only (Pre-Ninth-Grade) Dropouts 

Note: Figure does not include bilingual students or students with disabilities; these populations are traditionally 
excluded from public reporting on the ITBS. 
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STBS Math Scores Rise for Entering Ninth Graders, 
Not for ARC Students or Pre-Ninth-Grade Dropouts 
Last year of elementary school 




School Year 

r , APC Students and , ... , Ninth-Grade 

(Pre-Ninth-Grade) Dropouts Students Only 

Note: Figure does not include bilingual students or students with disabilities; these populations are traditionally 
excluded from public reporting on the ITBS. 



Fewer High Achieving Students Leave CPS 
between Eighth and Ninth Grades 




School Year 

ITBS Achievement in Seventh Grade: 

□ TopQuartile □ 3rdQuartile □ 2ndQuartile □ Bottom Quartile 



Note: We look at seventh grade performance on the ITBS because selective enrollment high schools use these 
scores for admissions decisions. 
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OUR EXAMINATION OF IMPROVEMENTS IN CHICAGO 
public high schools uses two sets of indicators: those that look at the full 
high school experience, and those that look at short-term outcomes. The 
first group includes graduation and dropout rates, percent completing a 
college preparatory curriculum, and percent passing at least one honors 
course over four years. 

Each is a cohort variable: for graduation and to enroll in college, enlist in the armed forces, and 



dropout rates, we track students starting at age 13, 
the year before the average student enters high 
school, until they exit the system either by gradu- 
ating, dropping out, or transferring to a school 
outside of the system; for the others, we track 
students from the year they become high school 
eligible over four years. We adjust for student 
characteristics on every measure to determine 
whether improvements, if any, might be attributed 
to reform in the high schools, or are a result of 
better-prepared students. 

The indicators that make up the short-term 
outcomes span fewer years. These include the 
percent of students on track for graduation, 
percent passing the algebra/geometry sequence by 
the end of tenth grade, and percent' passing an 
honors course in the first year. These short-term 
measures help us get a more current picture of 
student performance. As with the long-term 
measures, we adjust for student characteristics for 
each to see if improvements were simply the result 
of better-prepared students. 

Indicators of Basic Performance 

A high school diploma is the threshold require- 
ment for most post-secondary options; it is needed 

0 

ERLC 



secure higher-paying jobs. In comparison to 
graduates, dropouts fare significantly worse in 
terms of their employment eligibility, income 
potential, and likelihood of incarceration. 13 
Whether or not a student graduates or drops out is 
the most basic performance measure for high 
school. 

For our measures of graduation and dropout 
rates, we use the procedure outlined in the 
Consortiums 2001 report, Calculating a Cohort 
Dropout Rate } 4 We track a cohort of students for 
six years, from the fall of the school year they are 
13-years old (the age most students are in eighth 
grade) until they are age 19. In this way, we 
account for students who drop out between eighth 
and ninth grade, not just for those that enroll in 
high school. 

Graduation Rates Rise Slightly 

AJthough graduation rates remained low, they did 
improve slightly from 1993 to 2000 (see Figure 4). 
Of the CPS students who were 13-years old in the 
fall of the 1993-96 school year, 44.3 percent 
graduated by age 18 in the spring of 2000. This is 
an increase of 3.7 percent over the graduation rate 
for the 1992-93 cohort of 13-year-olds (40.6). 
Although we can track graduation rates to age 19 
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Figure 4 

Graduation Rates Rise Only Slightly after Adjusting for Changes in 
Student Composition 

Followed from age 1 3 




Cohort 

(13 Years Old in September of Cohort Year) 

Unadjusted Rate _ _ . Adjusted Rate _____ Unadjusted Rate ^ n n n , Adjusted Rate 

By Age 19 aDDD ' B y A g e1 g 1 By Age 18 □□□□> B y A g elB 



for only three cohorts, we do see an upward trend, 
from 51.0 percent for the 1992-93 cohort to 53.6 
percent for students who were 13 in the fall of 

1995. 15 

Even though this improvement is promising, 
the increase is cut in half when we adjust for the 
composition of incoming students. With an 
increase of only 1.6 percent, the upward trend in 
adjusted graduation rates by age 18 is substantially 
smaller than the unadjusted one. Similarly, the 
adjusted trends for graduation by age 19 show 
improvement of only 1.4 percent, compared to 

O 




2.6 for the unadjusted one. These adjustments 
indicate that only slight improvements in 
graduation rates can be attributed to changes in 
the high schools. 

Slight Decline in Dropping Out Primarily 
a Result of Better Prepared Students 

Although the overall dropout rate for Chicago 
public high schools remains above 40 percent, 
there was a slight decline over the period of our 
study. By the fall of 2000, 41.8 percent of the 
1995-96 cohort had dropped out of school. This 
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compares with 42.9 percent 
for the 1994-95 cohort and 
43.6 percent for 1993-94 (see 
Table 2). 16 

We could only calculate 
partial rates for the more recent 
cohorts. That is, we could 
follow students for two to five 
years, but not for six. Here we 
see a slight downward trend in 
dropping out by age 18 (from 
38.9 for the 1993-94 cohort to 
37.5 for 1996-97). Dropout 
rate trends for students under 
18 are less clear. They show 
neither the consistent pattern of 
decline that we see in the 



Table 2: Percent ©IF Students Dropping Out 
by Age and Cohort 



Dropped 
Out by 
Age 


Cohort 

(school year expected to enter high school) 




1993-94 


1994-95 


1995-96 


1996-97 


1997-98 


1998-99 


1999-00 


14 


4.4 


4.6 


4.4 


3.9 


3.9 


4.2 


4.0 


15 


8.8 


8.7 


8.0 


7.7 


7.7 


7.9 


7.3 


16 


18.0 


17.5 


16.8 


18.0 


18.1 


17.2 




17 


29.3 


29.4 


28.4 


29.3 


28.8 






18 


38.9 


38.6 


37.9 


37.5 








19 


43.6 


42.9 


41.8 










Final Year 
Fall 


1998 


1999 


2000 


2000 


2000 


2000 


2000 




complete cohort rates, nor a 
pattern of increases. 

Research has shown that 
retention makes students more 
likely to drop out in later 
years. 17 Since the systems 
adoption of the social promo- 
tion policy in 1995-96, CPS 
high schools have received larger 
percentages of students who 
were retained prior to ninth 
grade. In assessing the role of 
high schools on graduation and 
dropout rates, it is necessary to 
adjust these rates for students’ 
incoming achievement levels 
and also adjust for the propor- 
tion of students that were held 
back in eighth grade or in 
APCs. Therefore, we include 
whether a student was retained 
in eighth grade with our other 
controls for student characteris- 
tics. With this adjustment, we 
see only slight differences 
among the cohorts in dropout 
rates (see Table 3) . This indi- 
cates that the slight decrease in 
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Table 3: Adjusted Dropout Rates by Age and Cohort 



Dropped 
Out by 
Age 


Cohort 

(school year expected to enter high school) 




1993-94 


1 994-95 


1995-96 


1 996-97 


1997-98 


1998-99 


1999-00 


14 


4.4 


4.7 


4.5 


3.6 


3.2 


3.5 


4.0 


15 


8.8 


8.8 


8.2 


7.4 


7.4 


7.6 


7.6 


16 


18.0 


18.0 


17.5 


18.4 


19.0 


18.7 




17 


29.3 


30.2 


29.5 


30.8 


31.2 






18 


38.9 


39.5 


39.1 


39.4 








19 


43.6 


43.9 


43.1 










Final Year 
Fall 


1998 


1999 


2000 


2000 


2000 


2000 


2000 




To read Tables 2 and 3, follow one cohort down a column until its final 
year. A cohort is identified by when its members turn 13 years old — this 
is the year most students are in eighth grade. We begin tracking the 
cohorts dropout rate starting one year later, when members turn 14. 
The dropout rate is given as a percentage and followed down a column. 

Following the 1993 cohort in Table 2. In the fall of the 1993-94 
school year at age 14 — one year after we began following this cohort — 
4.4 percent had dropped out. The following year, at age 15, 8.8 percent 
had dropped out. By the fall of 1995 at age 16, 18.0 percent dropped 
out; 29.3 percent by the fall of 1996 at age 17; 38.9 percent at 18; and 
43.6 percent had dropped out by the fall of 1998-99 at 19. 
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dropout rates is mostly the result of better- 
prepared students. 

Indicators of Higher Levels of Student Performance 

Students’ performance in high school is equally as 
important an indicator as graduation and dropout 
rates. 18 Graduation rates describe only how many 
students meet the bare minimum of what we 
expect from high schools. To measure whether 
students meet more demanding expectations, we 
have created indicators of higher levels of school 
performance. One such indicator measures 
whether students are accumulating the appropriate 
credits to be accepted into a competitive college 
like the University of Illinois at Urbana- 
Champaign (UIUC). UIUC requires that entering 
students have four years of English, three years of 
math, three years of social science, three years of 
science, and two years of foreign language — the 
same requirements as those adopted by CPS in the 



1997-98 school year with its redesign of high 
schools. 

Another indicator of higher levels of perfor- 
mance is the percentage of students that attempt 
and pass an honors course during their high 
school careers. 19 Chicago public high schools offer 
more demanding courses for students who are 
willing and able to accept additional academic 
challenges. 20 While we have little guarantee that 
honors classes are of comparable quality from one 
school to the next, we are reasonably sure that 
within any one school, honors classes are more 
demanding than the general curriculum. We 
confirm the trend in honors course-taking by 
looking specifically at students who attempt and 
pass advanced placement (AP) courses. 

When looking at both of these indicators, we 
track our cohorts for four years, from the time 
they become high school eligible until the time an 
“on track” student would graduate. 
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Figure 5 



More Students Pass College Sequence Even after Adjusting for 
Changes in Student Composition 
Four years after elementary school 




Cohort 

Full Sequence: Unadjusted □□□□ Full Sequence: Adjusted 

Note: The full college sequence includes four English, three math, three science, three social science, 
and two foreign language credits. 



Small but Growing Number of Students Pass the 
College Preparatory Sequence 

Compared to all high school eligible students in 
the cohort we followed from 1993 to 1997, the 
percent of the 1996-2000 cohort that passed the 
full college sequence in four years rose from 9.6 
percent to 15.8, with the largest increase occurring 
in the final year (see Figure 5). One might argue 
that these numbers are artificially low because we 
include both dropouts and enrolled students who 
did not graduate in four years (students who 
transfer out of the system are not included). Of all 
four-year graduates, almost one-third successfully 
completed the college preparatory sequence (30.1 
percent), up from about one-fifth (12.1 percent). 
Nonetheless, although there has been substantial 
improvement on this measure, the vast majority of 
students still did not complete the sequence. 



When we adjust for student characteristics, the 
percent of students who became high school 
eligible in 1996 that took a full college sequence 
by 1999-00 is only 12.8 percent, up from 9.6 
percent for the 1993-97 cohort. This suggests 
that of the 6.2 percent increase in the unadjusted 
rate, 25 percent is due to the change in student 
composition and 75 percent is due to other 
factors (such as increased requirements). 

Completion of a moderated college preparatory sequence. We 

also looked at the percent of students who com- 
pleted the college sequence with the exception of 
the foreign language requirement. This is an 
academic program acceptable for admission to less 
selective colleges like Chicago State University. A 
higher and rising percentage of students met these 
requirements, from 15.2 of the 1993-97 cohort 
up to 23.5 percent of the 1996—00 cohort (see 
Figure 6). Among CPS students who graduated in 
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2000, we see the biggest jump with a full 45.2 
percent meeting these reduced requirements. Even 
after adjusting for changes in student composi- 
tion, we still see a 5.6 percent increase, up from 
31.6 for the 1993-97 cohort. Again, this increase 
suggests that most of the improvement is not 
simply a result of changes in enrollment. 

More Students Pass an Honors Course in Four Years 

Over the period of our study, a substantial and 
rising percentage of CPS students took and passed 
honors courses at some point during their high 
school careers (see Figure 7). 21 For the most recent 
cohort (those who became high school eligible in 
1996 and expected to graduate in 2000), over one- 
fourth of students (25.9 percent) took and passed 
at least one honors course in four years, up slightly 
from 22.9 percent. Of the students from this 
cohort who did graduate within four years (rather 
than transfer, drop out, or not graduate), 44.3 
percent took and passed at least one honors 
course. 22 

When we adjust for student composition, there 
is virtually no change between the percent of 



students in the 1993-97 cohort who passed an 
honors course and the percent of the 1996-00 
cohort that passed. The adjusted rate for all 
cohorts hovered at around 22 percent. This 
indicates that the rising percentage of students 
taking an honors course is due primarily to 
better-prepared students. 

A similar pattern with AP courses. Because the meaning 
and challenge of honors courses can vary widely 
across schools, we look in particular at the trend of 
students taking and passing AP courses. AP 
courses are more elite than honors classes and are 
aligned to the national AP exam. We found a 
similar pattern: a slight increase in the number of 
students who attempted and passed an AP course 
(from 6.4 percent of the 1993-97 cohort to 8.0 
percent for the 1996-00 one). Adjusted trends 
show no improvement, remaining steady at about 
6.5 percent. Since 98.9 percent of all students who 
passed an AP course either transferred out of the 
system or graduated, looking only at the students 
who graduated from a CPS high school raises this 
percent, from 13.3 percent for the graduating class 
of 1997 to 15.2 for the graduating class of 2000. 
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More Students Pass College Sequence without Foreign Language 
Even after Adjusting for Changes in Student Composition 
Four years after elementary school 




Note: The partial college sequence includes four English, three math, three science, and three social science credits. 



Slightly More Students Pass Honors Course, hut not after 
Adjusting for Changes in Student Composition 

Four years after elementary school 





Note: Honors courses include honors, telescoped honors, International Baccalaureate, and advanced 
placement courses. 



Figure 6 



Figure 7 
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THE INDICATORS WE HAVE LOOKED AT THUS FAR 
describe student performance over the course of a full high school career. 
One disadvantage of these cumulative measures is that because they take 
place over a long time period they are relatively insensitive to even large 
changes in policies and practices. 



To understand more current trends, therefore, we 
must also look at short-term outcomes. These 
include students’ success in their first year after 
elementary school and whether they are passing 
more challenging coursework. 

Here Students On Track to 
Graduate after ini Grade 

Students’ performance in their first year after 
elementary school is critical to the overall success 
of their high school careers. We created a baseline 
indicator of first-year success that determines 
whether a student is “on track” or not. To be on 
track, a student must earn enough credits to 
assume sophomore status on time and have 
received no more than one failing grade in a 
core course (English, math, science, and social 
science). Students who are sent to APCs or 
drop out are automatically off track. Being on 
or offtrack is highly correlated with long-term 
performance; students who are off track after their 
first year have tremendous difficulty catching up 
and graduating within four years. In the 1996-97 
school year, 78 percent of high school eligible 



students who were on track one year after elemen- 
tary school graduated within four years. Only 15.6 
percent of off-track students graduated within this 
same period. 

The on-track measure is sensitive to changes in 
its two key components: how many courses 
students take, and how well they do in them. By 
counting credits we learn whether students take 
and pass an adequate course load to graduate 
within four years. We identify unacceptable 
performance by counting the number of failing 
grades students receive in core courses. Together, 
we get a baseline indicator of acceptable (but not 
necessarily strong) academic performance for high 
school eligible students. 

Despite Steady Improvement Only Half of Ninth 
Graders On Track in 1999-00 

In the 1999-00 school year, 50.6 percent of all 
high school eligible students were on track to 
graduate within four years. Although this 
percentage is low, it is still a substantial im- 
provement since 1993—94, when only 42.2 
percent of high school eligible students were on 
track (see Figure 8). 
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Figure 8 

More Students On Track Even after Adjusting for Changes in 
Student Composition 

One year after elementary school 




School Year 

□ Unadjusted □ Adjusted 
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After adjusting for changes in 
student composition, 5.4 
percent more students were on 
track (from 42.2 percent in 
1993-94 to 47.6 percent in 
1999-00). This suggests that 
slightly less than half of the 
improvement in on-track rates 
is the result of better-prepared 
students entering high 
schools; more than half is due 
to other factors. 

The improvement in on-track 
rates is the result of two factors: 
students enrolled in many more 
courses and they passed them at 
a slightly higher rate (see Figures 
9 and 10). In 1993-94, the 
average student attempted 4.9 
credits, slightly under the five 
needed to matriculate to tenth 
grade. With the systems high 
failure rates, students received 
even fewer credits. By the 1 999 — 
00 school year, students at- 
tempted an average of 5-7 
credits. This increase, coupled 
with a decrease in failure rates, 
resulted in more students 
being on track. In the 1 999 — 

00 school year, the average 
student received 4.6 credits, 
up from 3-7 in 1993-94, 
almost a full course more. 

Off-Track Students Have 
Difficulty Recovering 

Can off-track students get back 
on track? Not very easily. For 
students who were off track after 
their first post-elementary 
school year in 1993-94, only 
6. 1 percent were able to get 
back on track by the end of the 




Figure 9 

Students Attempt and Receive More Credits 
One year after elementary school 



75.5% 74.0% 76.0% 78.4% 77.2% 77.2% 80.7% 

pass rate pass rate pass rate pass rate pass rate pass rate pass rate 




1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 1999-00 

School Year 



□ Attempt □ Receive Full Course Load 

Note: Five credits is a full course load for a first year high school student. 



tenth grade — or had enough 
credits to matriculate to the 
11th grade and received no 
more than one failing grade in a 
core course that year (see Figure 
1 1). This number rose to 8.9 
percent for students off track in 
1997-98 (starting with the 
summer of 1997 we include 
credits earned in summer 
school), but dropped again to 
6. 1 after the number of credits 
required to move to the 1 1th 
grade increased from 10 to 1 1. 
In general, students remained 
off track because they did not 
have enough credits rather than 
because of high failure rates. 

28 



Percent of Students Pursuing More 
Challenging Course Work Rises 

As with long-term indicators of 
moderate and advanced student 
performance, the percent of 
students pursuing more chal- 
lenging course work on similar 
short-term measures also rose 
from 1993 to 2000. More high 
school eligible students at- 
tempted and passed the algebra/ 
geometry math sequence by the 
end of the tenth grade, and 
more attempted and passed an 
honors course their first year 
after elementary school. This 
suggests that the long-term 
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Figure 10 

Passing Rates Rise 

One year after elementary school 




1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 1999-00 

School Year 



□ Foreign Language D English H Science D Social Science H Math 



trend of students passing honors courses and 
completing a college preparatory curriculum over 
their full high school careers will continue to rise. 

More Students Passing Algebra/Geometry Sequence 

The algebra/ geometry sequence has been docu- 
mented in research literature as a gatekeeper for 
college enrollment. 23 From 1993 to 2000, there 
was a steady increase in the percent of CPS stu- 
dents attempting and passing these courses, 
especially after they were made a requirement for 
graduation in the 1997-98 school year (see Figure 
12). 24 In the 1993-93 school year, 36.3 percent of 
all high school eligible students passed the 

O 




sequence two years after graduation from 
elementary school. By 1998-00, that percent 
had risen to 49.7. 

When we adjust for the characteristics of 
incoming students, the trend is still up substan- 
tially. For the 1998-00 cohort, 50 percent of 
students passed the sequence, compared with 37 
in 1993—95. This suggests that changes in 
incoming students account for only 17.8 percent 
of improvement, while other changes, possibly 
curricular reforms in the high schools, account 
for the bulk of the improvement (82.2 percent). 
As we will show later in this report, this rise is 
consistent with improved student performance on 
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Few Gffff-Tirack Sttimdeirotts Gett Back On Track 
Two years after elementary school 
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1993-94 1994-95 1995-96 1996-97 1997-98 1998-99 



School Year 

E! Fall and Spring Credits Only □ Summer Credits Included 

Note: In the 1 998-99 school year, credit requirements increased from 1 0 to 1 1 for sophomores. 
Summer school transcripts became available only in 1995-96 



Figure 1 1 



Figure 12 



Moire Students Pass Algebra and Geometry Even after 
Adjusting for Changes on Student Composition 
Two years after elementary school 
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Over the course of this study, high school eligible students were taking and passing more courses in 
core subjects. However, in English and math, arguably the subjects with the most emphasis, there 
was actually a slight decline in the percent of students attempting courses and only a slight increase 
in the percent passing them. The biggest increase was actually in social science and science courses. 
In the 1993-94 school year, only 55 percent of students attempted a social science course and 35 
percent passed. By 1999-00, 75 percent of students, a full 20 percent more, attempted a social 
science course and 53 percent passed it. The trend for science is similar — in the 1993-94 school 
year, 62 percent of students attempted a science course and 42 percent passed; that percentage rose 
to 78 percent attempting and 56 percent passing in 1999-00. 

Although the improvement in course-taking trends began before the imposition of the new 
graduation requirements, there was a substantial bump in 1997-98 when they went fully into effect. 
Despite the new requirements, however, there was virtually no change in the percent of high school 
eligible students attempting foreign language classes in their first year, which hovered around 12 
percent for the entire period of our study. Since this is only a two-year requirement, however, not 
taking a foreign language class in a students first year after elementary school is not necessarily 
problematic. See Appendix III for figures showing the percentage of first-year students taking core 
courses from 1993 to 2000. 



the math section of the TAP. While there is always 
the possibility that schools are changing the names 
of math courses without actually changing the 
content, the increase in adjusted TAP scores 
suggest some real gain in mathematics learning in 
high schools (see Figure 15 on page 30). 

Although this improvement is substantial, for 
students graduating by the year 2001, both the 
algebra/geometry sequence and advanced algebra/ 
trigonometry are requirements for graduation. In 
this light, a 51 percent passing rate still means that 
half of all high school eligible students are behind 
in their math requirements two years after elemen- 
tary school. 

Number of Students Passing an 
Honors Course Rises Substantially 

Earlier we saw an upward trend in students 
enrolling in honors courses across a standard high 
school career (see Figure 7 on page 19). Here too 



there is a similar trend. In the first three years of 
our study, about 8.4 percent of first-year students 
passed at least one honors course. In 1996-97 
however, this increased to 10.4 percent, and it 
continued to rise for the next three years (see 
Figure 13). By 1999-00, 15.4 percent had passed 
an honors course. This corresponds to the increase 
in students attempting these courses, which rose 
from 9.2 percent in 1993-94 to 16.6 percent in 
1999-00. 

Once we adjust for changes in student composi- 
tion however, we see only a small increase in the 
percent of students passing an honors course, from 
8.4 percent in 1993-94 to 9.6 in 1999-00 (see 
Figure 13). From these more modest gains, we 
can assume that most of the growth is the result 
of the demographics of incoming students, 
particularly their higher ITBS scores, rather than 
changes in the high schools. 
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Figure 13 

More Students Pass At Least One Honors Course Even after 
Adjusting for Changes in Student Composition 

One year after elementary school 





What Percent of Students Attempt and 
Pass Algebra and Geometry? 

Two years after elementary school 




School Year 

□ Attempt □ Pass 



The increase in students passing algebra and 
geometry seems driven by an increased 
number of students attempting the se- 
quence, possibly as a result of the new 
graduation requirements. From the 1993-95 
school year to 1995-97, the attempt rate 
hovered at around 52 percent. In 1996-98, 
however, it began to rise, and by 1998-00 
nearly two-thirds of all students attempted 
the sequence. 
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EVEN AFTER WE ADJUST FOR THE CHARACTERISTICS OF 
students taking the Tests of Achievement and Proficiency (TAP), we 
still find marked improvement over time. Standardized TAP scores 
allow us to compare the performance of CPS students to other stu- 
dents across the country. 



Nationally, the average grade equivalent (GE) 
score for the ninth grade is 9.8. A student with 
this score performs like an average ninth grader in 
the eighth month of the school year. Among CPS 
ninth graders who took the test, there was enor- 
mous improvement from 1993 to 2000, especially 
after the 1997-98 school year (unlike the other 
indicators in this report, we examine TAP score 
trends for ninth-grade students only, not for all 
students leaving elementary school). 




TAP Scores rose substantially after the implemen- 
tation of the promotion policy in 1996. Average 
scores in reading improved from 8.2 GEs in 
1993-94 to 9.8 GEs in 1999-00. In math, scores 
rose from 8.3 GEs in 1993—94 to 10.0 in 1999- 
GO (see Figures 14 and 13). Chicago’s average is 
now comparable to the national average in reading 
and slightly above the national average in math. 

About Half of the Improvement 
Due to Better Prepared Students 

Even after adjusting for differences in student 
composition over time, we still find a significant 



upward trend. In reading, adjusted scores rose 
from 8.2 to 9.1, or 0.9 GEs, and in math they rose 
from 8.3 to 9.1, or 0.8 GEs. This amounts to 
nearly a years additional learning in both subjects. 
Both unadjusted and adjusted trends show 
strong improvement in ninth graders’ basic 
skills. This indicates that only part of the 
improvement from 1993 to 2000 is a result of 
the enrollment of better-prepared students in 
high schools. 

Was Improvement Affected by Excluding Students? 

Our adjusted trends suggest that about half of the 
improvement in student achievement is due to 
changes in the characteristics of students entering 
ninth grade. We can see this directly when we look 
at the percent of students taking the TAP who 
scored in each ITBS quartile in eighth grade. Here 
we see that while the percent of students who took 
the TAP who had ITBS scores in the top three 
quartiles increased, the percent from the bottom 
quartile fell from 32.8 to 17.1. This decline began 
immediately after the implementation of the 
promotion gate policy in the 1993-96 school year 
(see Figure 16). Absolute numbers declined in a 
similar fashion, with a drop of over 3,000 students 
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Figure 14 

Ninth-Grade TAP Reading Scores Rise Even after Adjusting for 
Changes in Student Composition 




Figure 15 

Ninth-Grade TAP Math Scores Rise Even after Adjusting for 
Changes in Student Composition 





School Year 

a □ b a 0 Adjusted Math Score 



i i Unadjusted Math Score 
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Means Versus Medians 

This report uses average TAP scores over time. CPS reports medians. Although the scores are lower when 
using TAP median scores to assess change over time, the overall trend remains the same. Because some 
year-by-year increases in TAP scores may be due to variations in the test form, these figures compare the 
years in which students took the same form of the test. Looking across time by test form (color-coded on 
the graphs), we still see strong gains over time. 

Ninth-Grade TAP Median Reading Scores Rise 



eighth-grade high school 

promotional redesign 

gate 
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from the lowest quartile and increases in the other 
three quartiles (see Appendix IV). 

The improvement in TAP scores does not 
appear to be a result of high schools discouraging 
more (low achieving) students from taking the 
TAP. The percent of first-time ninth graders whose 
TAP scores are included for reporting actually rose 
slightly, from 66.4 to 68.8 percent between the 



1993-94 and 1999—00 school years (see Figure 
17). Although there was a slight increase in the 
percent of students not taking the TAP at all 
(from 20.2 to 21.0 percent), there was a bigger 
decrease in the percent of students who took the 
TAP, but were not included in reporting (from 
13.4 to 10.2 percent). 



Figure 16 



Fewer Entering Ninth Graders Score in Bottom Quartile 
on Eighth-Grade Reading 
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ft/lore First-Time Ninth Graders Take the TAP 
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FROM 1993 TO 2000, STUDENT PERFORMANCE IN CPS 
high schools improved, if only slightly, on every indicator. At the same 
time, this improvement did not occur uniformly across high schools — 
some saw dramatic improvement while others continued to lag behind. 
Was there a pattern of success among the different types of schools? 



In recent years, there has been a proliferation in 
high school choices for students. We compared 
TAP scores and two-year dropout rates for six 
different types of schools that existed in the 1999— 
00 school year: extended elementaries, charter and 
small schools, selective admissions, schools on 
probation, APCs, and neighborhood and voca- 
tional schools. 25 

TAP scores provide a measure of students’ basic 
skills in reading and math by the end of the ninth 
grade. It is important to recognize, however, that 
the proportion of students who make it through 
ninth grade affects the average scores for each 
school. Schools with higher proportions of stu- 
dents dropping out prior to the TAP will have 
higher scores than similar schools where these 
students remain. To balance this, we also examine 
two-year dropout rates. For each measure, we first 
look at the unadjusted performance of each 
school. Then we look at the value added by each 
school in each category. Because schools serve very 
different populations of students, we control for 
student characteristics to determine the average 
amount each school adds to its students’ TAP 
scores and the school’s impact on the likelihood of 
a student dropping out within two years. 




The observed (unadjusted) reading and math TAP 
scores by school category are not too surprising. 
(See page 40 for a description of school catego- 
ries.) We see that selective admissions schools had 
the highest overall TAP scores and probation 
schools had the lowest (see Figures 18 and 19). 

But school means tell us little about how much 
schools contribute to students’ learning. Students 
that enroll in probation schools are not always 
very well prepared, whereas selective admissions 
schools receive the academic cream of the crop. 

To examine the value added by high schools, we 
created a model that shows us how well schools 
performed in comparison to what we would 
expect given the characteristics of their incoming 
students. The most important factor in this 
adjustment was students’ incoming ITBS scores. 
(See Appendix II for a full discussion of how the 
adjustments were made.) The average value added 
of all the schools is zero. Therefore, if a school had 
a score above zero, it performed better than 
expected given the population of students it 
served; if it had a score below zero, it performed 
worse (see Figures 20 and 21). While all of the 
schools together have an average score of zero, 
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Figure 18 

1999-00 TAP Reading Scores for Ninth Graders 

Not adjusted for any change in student composition 
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Figure 19 

>1 999.00 TAP SViatth Scores ffor Ninth Graders 
Not adjusted for any change in student composition 



grade level (9.8 GEs) 
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Figure 20 

1 999-00 TAP Reading Scores for Ninth Graders 

Adjusted for changes in student composition 
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Figure 21 

1999-00 TAP Math Scores for Ninth Graders 

Adjusted for changes in student composition 
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Extended Elementary Schools are grade schools that include a ninth grade; they generally enroll freshmen 
from their own eighth-grade classes. Extended elementaries are meant to ease students’ transition to high 
school. 

Charter Schools are exempt from many of the restrictions placed on regular CPS high schools, but they are 
not allowed to use selection criteria in their admissions. Each school develops its own mission and serves 
somewhat different populations. 

Selective Admissions High Schools were established to attract and retain the best students in the system. 
Enrollment is academically selective. 

Probation High Schools are neighborhood (general admission) high schools placed on probation because 
the bulk of their students (85 percent or more) failed to score at or above national norms. Even though 
these schools are under the threat of sanctions, they also receive extra resources to finance improvement 
efforts and facilitate partnerships with external partners. 

Academic Preparatory Centers (APCs) prepare very low scoring eighth graders for high school. Although 
APC students do not take the TAP, we can compare their two-year dropout rates. 

Neighborhood High Schools serve the general population and primarily enroll students from their own 
geographic area. Vocational High Schools have a specific mission to prepare students for a particular career. 
They also generally enroll students from the surrounding area. These two types of schools are combined in 
our analyses. None of these schools were on probation in 1999-00. 

Note: Some schools may use criteria for admission that are not controlled for in our analyses, such as recommenda- 
tion letters and elementary school attendance. In these cases, their students’ outcomes may appear better than those of 
students in other schools simply because of this selection bias. 



categories of schools can be positive or negative. 
For example, Hyde Park Academy’s unadjusted 
ninth-grade TAP reading scores were above the 
norm for ninth graders at 10.0 GEs, but they were 
actually 0.6 GEs lower than we would expect 
given the students enrolled there. By comparison, 
Crane High School’s average TAP reading scores 
were 0.6 GEs below the national norms at 9.2, but 
they are 0.6 GEs higher than we would expect 
given the students it enrolled. 

For reading, schools’ value added ranged from 
0.8 GEs lower than we would expect given their 
incoming population (North Lawndale High 
School), to 1.2 GEs above the expected score 



0 




(Westinghouse). In math, value added ranged 
from nearly 1.0 GEs below expected performance 
(Washington), to 1.5 GEs above (Marshall). 

Many of the schools whose value added was 
substantially worse than expected test a higher 
percentage of their students than those with a 
high value added. For example, North Lawndale 
tested 100 percent of its students in reading and 
99 percent in math, while Marshall only tested 
89 percent of its students in each subject (see 
Table 4). 

After adjusting for student background, each 
type of school showed mixed results. Nevertheless, 
there are some notable findings within each of the 
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Table 4: Percent of Students Taking the TAP in 2000 



School 


Reading 


Math 


School 


Reading 


Math 


Extended Elementary Schools 






Hirsch 


86 


86 


Hope Community 


96 


98 


Kelvyn Park 


78 


74 


Lozano Bilingual 


100 


100 


Manley 


81 


81 


Chavez 


79 


79 


Marshall 


89 


89 


Seward 


92 


92 


Phillips 


82 


81 


Douglass 


84 


80 


Senn 


85 


82 


Dyett Middle 


90 


88 


South Shore 


84 


80 


Charter/Small Schools 






Tilden 


74 


74 


Best Practices 


89 


89 


Wells 


85 


83 


North Lawndale 


100 


99 


Spaulding 


96 


96 


Hancock 


95 


95 


Englewood 


81 


81 


Future Commons 


94 


94 


Orr 


81 


81 


ACT 


91 


91 


' Clemente 


88 


87 


Nuestra America 


86 


86 


Carver 


86 


83 


Noble Street 


94 


94 


Corliss 


85 


87 


Perspectives 


100 


100 


Juarez 


79 


79 


Chicago International 


64 


64 


Neighborhood Schools 






Selective Schools 






Chicago Vocational 


89 


89 


Jones 


100 


100 


Prosser 


96 


95 


Lane Technical 


96 


96 


Simeon 


93 


93 


Lindblom 


92 


93 


Amundsen 


84 


84 


Southside Prep 


94 


94 


Bogan 


87 


87 


Von Steuben 


94 


94 


Bowen 


76 


76 


Northside Prep 


97 


98 


Foreman 


81 


81 


Chicago Agricultural 


99 


99 


Hyde Park 


94 


94 


Chicago Military 


97 


97 


Kelly 


77 


76 


Young Magnet 


98 


98 


Kennedy 


91 


91 


Probation Schools 






Lake View 


91 


89 


Dunbar 


92 


92 


Mather 


88 


88 


Flower 


90 


90 


Morgan Park 


95 


95 


Richards 


88 


90 


Roosevelt 


82 


81 


Westinghouse 


95 


96 


Schurz 


84 


84 


Austin 


78 


78 


Steinmetz 


86 


86 


Calumet 


72 


72 


Sullivan 


76 


75 


Crane 


84 


83 


Taft 


85 


84 


DuSable 


82 


82 


Lincoln Park 


90 


91 


Farragut 


81 


81 


Washington 


84 


85 


Fenger 


84 


84 


Hubbard 


94 


94 


Robeson 


86 


86 


Kenwood 


92 


92 


Gage 


87 


87 


Curie 


94 


94 


Harlan 


84 


82 


Julian 


83 


83 


Harper 


82 


81 


Collins 


82 


84 
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groupings. In many selective admissions schools, 
ninth graders scored even higher than we would 
expect given their incoming students’ already 
advanced abilities. Most of the newly created 
regional magnets performed especially well. 
However, Chicago Military Academy performed 
worse than expected, especially in math, given the 
background of its ninth graders. 

The value added by probation schools was 
higher than for some neighborhood schools, after 
taking differences in incoming population into 
account. In Figure 21, when we look at the 
number of schools with better than expected 
achievement (those to the right of the line) and 
those with worse than expected achievement 
(those to the left of the line), we see that over half 
of the probation schools performed better than 
expected in reading and about half of them did so 
in mathematics. Just over a third of neighborhood 
schools showed better than expected scores in 
reading or math. Many charter and small schools 
performed somewhat worse on the TAP than one 



would predict given their student population (see 
Appendix IV). 

The extended elementaries had mixed results, 
suggesting that the format worked effectively in 
some schools but not in others. The evidence 
certainly does not indicate that keeping ninth 
graders in the smaller, more protected environ- 
ment of elementary education is a foolproof 
method of increasing learning. The mixed results 
within each category demonstrate that none of the 
models give us a clear blueprint. Rather, the 
particular workings of individual schools account 
for their success or lack thereof. 

Dropout Rates 

Turning to two-year unadjusted dropout rates, we 
find APC dropout rates were extremely high (see 
Figure 22). Extended elementaries, neighborhood 
schools, and schools on probation had mixed 
results, although probation schools reported a 
higher dropout rate than other neighborhood 
high schools. Selective admissions schools had 
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very low dropout rates, and charter 
schools’ rates were also low. 

Adjusted dropout rates present a very 
different pattern than the TAP scores after 
we control for the populations the schools 
served. Some with relatively high absolute 
dropout rates actually kept more students 
enrolled than we might expect once we 
take into account the characteristics of 
their incoming students. For example, 

Hayes APC had a two-year dropout rate 
of nearly 44 percent, but it actually kept 
about 6 percent more students than we 
would expect (see Figure 23). Similarly, 
some schools with low dropout rates had 
slightly higher dropout rates than we 
would expect given the population they 
enroll. For example, Lincoln Park had a 
two-year dropout rate of about 13 
percent, but had about 12 percent more 
students drop out than we would 
expect. (Please note that a lower drop- 
out rate is better.) 

One of the most noticeable patterns is 
that most selective admissions schools and 
most charter and small schools had lower 
dropout rates than we might expect given 
their student population. This is especially signifi- 
cant for the charter and small schools, many of 
which have made preventing students from 
dropping out part of their core missions. It is also 
noteworthy that while the APCs as a group had 
extremely high overall dropout rates, some of the 
APCs, especially Hayes, Partee, and Dorsey, kept 



more students enrolled than we might expect. 
Neighborhood and probation schools showed a 
somewhat mixed set of outcomes, although 
probation schools did not look significantly worse 
than other neighborhood schools once we con- 
trolled for their student composition. 
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Figure 22 

Two-Year Dropout Rates: Fall 1998 to Fall 2000 

Not adjusted for any change in student composition 



Hayes APC 
Dorsey APC 
Partee APC 
Sengstacke APC 
Proctor APC 
Rodriguez APC 
Hernandez APC 
Olive APC 
Harvey APC 

Douglass 

Hope 

Lozano 

Seward 

Chavez 

North Lawndale 
ACT 
Hancock 
Future Commons 
Best Practices 
Chicago International Longwood 
Chicago International Bucktown 
Nuestra America 
Perspectives 

Jones 
Lane Tech 
Chicago Agricultural 
Lindblom 
Young 
Southside Prep 
Von Steuben 

Spalding 
Hirscn 
Wells 
Clemente 
Crane 
Marshall 
South Shore 
Calumet 
Orr 
Juarez 
Fenger 
Richards 
Englewood 
Gage Park 
Flower 
Westinghouse 
Robeson 
Dunbar 
Kelvyn Park 
Phillips 
Harlan 
Corliss 
Manley 
Harper 
Farraaut 
Du Sable 
Tilden 
Senn 
Carver 
Austin 

Mather 
Chicago Vocational 
Taft 
Foreman 
Kelly 
Simeon 
Prosser 
Collins 
Bogan 
Lake \/iew 
Kennedy 
Amundsen 
Julian 
Morgan Park 
Stein metz 
King 
Curie 
Schurz 
Hubbard 
Hyde Park 
Washington 
Bowen 
Kenwood 
Sullivan 
Roosevelt 
Lincoln Park 




0% 



10% 20% 



30% 40% 50% 



60% 




49 



Student Performance 



45 



Figure 23 

Two-Year Dropout Rates: Fall 1998 to Fall 2000 

Adjusted for changes in student composition 
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VII. A Closer Look at Academic Preparatory Centers 



WHEN ACADEMIC PREPARATORY CENTERS (APCs) WERE 



established in the 1997—98 school year, there were conflicting opinions 
about the effects they would have. Their purpose was to help transition to 
high school those students who were too old for elementary school but 
had yet to meet the ITBS achievement thresholds required for graduation 
from eighth grade. 



Ideally, these students would benefit from a 
curriculum specially designed to prepare them for 
the rigors of high school (somewhat like the 
Summer Bridge program in elementary school) 
and students who enrolled directly in high school 
from the eighth grade would benefit from a more 
challenging classroom environment. 26 On the 
other hand, some worried that APCs would only 
be a “cooling out” place for low-performing 
students to go until they tired of school and 
dropped out. Our evidence suggests that APC 
performance fell somewhere in the middle of these 
opposing views. 

What is the Typical Educational Experience 
of APC Students? 

Approximately 1,600 students enroll in APCs for 
the first time each year. 27 The curriculum focuses 
primarily upon remedial math and English; 
students do not receive high school credit for this 
coursework, although other classes are offered for 
credit. In the 1999-00 school year, the average 
APC student enrolled in 2.9 credit-granting (non- 
remedial) courses and earned 2.2 credits — about 
half of the typical course load for a first-year high 



school student (see Figure 24; for a comparison to 
all high school eligible students, see Figure 9 on 
page 23). Less than 3 percent attempted a full 
course load and over 63 percent failed at least one 
course. 

In 1999-00, only 17 percent of APC students 
attempted high school English and 26 percent 
attempted high school math (the rest generally 
enrolled in remedial math and English classes, 
which are not offered for credit). Very few enrolled 
in science or foreign language courses — less than 5 
percent each year. The only high school-level core 
course that was common among APC students 
was social science, 70 percent attempted it in 
1999-00. For the low percentage that took high 
school core courses, an even smaller percent passed 
(see Figure 25; for a comparison-to all high school 
eligible students, see Appendix III on page 60). 

Dropout Rates Especially High 

Since their establishment in 1997-98, roughly a 
quarter of all APC students drop out in their first 
year. For the 1997 cohort, the dropout rate was 28 
percent; 25 percent for the 1998 cohort; and 24 
percent for 1999. Two years after the 1997 cohort 
enrolled in an APC, 48 percent had dropped out 
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Figure 24 

APC Students Do Not Attempt or Pass a 
Full High School Course Load 




School Year 

IH Attempt □ Receive Full Course Load 

Note: Five credits is a full course load for a high school student 



(see Figures 26, 27, and 28). 
This compares to a 15 percent 
dropout rate for the 1998-99 
cohort of all students who 
entered ninth grade. Still, 
some of the APCs had fewer 
dropouts than we would 
expect given the characteristics 
of their incoming students (see 
Figure 23 on page 45). 28 

Enrollment in High School 

During the period of our study, 
about half of first-time APC 
students went on to high school 
after one year; 41 percent of the 
1997-98 cohort, 58 percent of 
the 1998-99 cohort, and 57 
percent of the 1999-00 cohort. 
Correspondingly, in each of 
these three years, a substantial 
group that did not drop out also 
failed to make enough progress 
to move on to high school. 
These students were retained in 
APCs for a second year. In 
1997-98, 22 percent of students 
were retained after one year and 
1 1 percent were retained in each 
of the following two years. 



How Do APC Students 
Perform in High School? 

When APC students did enroll 
in high school they had diffi- 
culty getting and staying on 
track. Of the 1997-98 cohort 
that enrolled in a CPS high 
school after one year in an APC, 
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Progress off First-Time APC Students: Starting Fall 1 997 Progress off First-Time APC Students: Starting Fall 1 998 



50 



THE STATE OF CHICAGO PUBLIC HIGH SCHOOLS: 1993 to 2000 



(N 



3 

to 

£ 



VO 

(N 





OJ 

c 

3 



C 

<u 

U 

Q. 

a* 

i— 

Q_ 

U 

I 

QJ 

T3 

U 

< 

c 

TJ 

_aj 

"5 

k. 

c 



o 

o 



CJ) 

X 

1/1 

Q_ 

u 

TJ 

_aj 

"5 

k_ 

c 



zj ■ 
O 
a. 
o 



c 

o 



u 

«i= 



u 

k_ 

_fU 

3 

cr> 

<u 



aj 

> 

fD 

<v 



CL O 

-O c 
^ ai 
o 

c OJ 
<X> Q- 



ro 



uo 



CL Of 

§ B 

If aj 
cn xi 

.1 i 

Xj f= 

fO cn 

ai c 

O D 
"O to 
C 

ro O 
1/1 £ 

Pj <u 
u 

gJ ai 
% Q. 

_gj 

+J c 
3 J9 
° ■£ 
o V. 
* 1 



o o o 





1999 2000 2001 2002 



Student Performance 



49 



only 30.8 percent were on track 
by the end of their freshman 
year. 29 Students from the 1998- 
99 cohort that enrolled in high 
school in 1999-00 attempted 
6.1 credits their freshman year 
(more than they needed to 
achieve sophomore standing), 
but received credit for only 4.0. 
At 63.6 percent, their passing 
rate was considerably lower 
than that for all high school 
eligible students, which was 
80.7 percent in 1999-00 (see 
Figure 29). 

On average, former APC 
students did attempt a full 
course load once they enrolled 
in high school, but were more 
than a full credit short of 
officially passing out of ninth 
grade (see Figure 30). Further 
analyses show that those stu- 
dents who were on track at the 
end of their first year were able 
to do so primarily because they 
had already earned some high 
school credits in the APC and in 
summer school. This raises 
questions about whether these 
students will be able to remain 
on track in later years. Only 
after two full school years (one 
in an APC and one in high 
school) and one summer were 
these students able to earn 
enough credits to be on track 
after their first year. 



Figure 25 

What Percent of APC Students Attempt and 
Pass High School Core Courses While 
in an APC? 




Percent of students 



E Attempt Q Pass 

Note: Numbers for 5 percent or less are not shown. 
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Former APC Students Attempt but 

Do Not Pass a Full Course Load 

Students promoted to high school after one year 



63% 66% 




□ Attempt □ Receive Full Course Load 



What Courses Do Former APC Students Attempt and Pass? 

Students Promoted to high school after one year 




English Math Social Science Science Foreign 

Language 




□ Attempt □ Receive 



Figure 29 



Figure 30 
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IN THIS REPORT WE EXAMINE THE PERFORMANCE OF CPS 
high school students from the 1 993—94 school year to 1999-00 on a 
variety of measures. Overall, outcomes improved — greater numbers of 
students graduated, stayed on track after their freshman year, and passed 
a more challenging course load. 



We found, however, that much of these improve- 
ments were the result of students leaving elemen- 
tary school better prepared for high school. More 
high achieving students were staying in the system 
for high school. And, at the same time, thousands 
of low achieving students were being retained in 
eighth grade. Other than improvements in student 
course-taking spurred by the revision of 
systemwide graduation requirements, no high 
school reform had any discernable effect. 

Despite real progress, there is still concern 
about the quality of CPS high schools. Even in 
light of the improvements noted above, student 
performance remains generally low. Barely 50 
percent of students graduated or even got started 
on the right foot in their first year. Nor did 
sufficient numbers actually pass the more chal- 
lenging courses required by CPS — even by the 
1999-00 school year, not quite half of students 
passed the algebra/geometry sequence within two 
years, and fewer than one-quarter passed the 
college preparatory sequence within four. 

What can we learn from these results to im- 
prove student performance in the future? First, 
preparing students well before they enter high 
schools is clearly of critical importance. Success in 



the elementary schools breeds success in the high 
schools. Second, the only provision of the 1997 
high school redesign that can be clearly associated 
with improvement in student performance were 
policies that mandated a more rigorous curricu- 
lum, emphasizing the difficulty of changing 
performance through systemwide initiatives. 

Third, school-by-school analyses show that there 
was some success in each type of high school. This 
suggests that it is individual schools rather than 
any particular type of school that play the 
biggest role in whether students exceed expecta- 
tions (although charter and small schools do 
appear to have especially low dropout rates 
given their student populations). We must look 
more closely at successful schools to see why 
they are making a difference. 

This report establishes a baseline for important 
indicators, a baseline that takes into account 
changes in the characteristics of high school 
students. Going forward, we must continue to 
monitor these indicators and chart the progress of 
high school reforms. In future work, we will 
explore the environment inside high schools to see 
why some do better than others, taking into 
account their students’ level of preparation. 
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Sample 


Reason 


Graduation Rates and 
Movement Through the 
System 


Students who were age 1 3 
in September 


Sample is not affected by changes from the 
retention policy. 1 


Cohort Course Outcomes 


Students who were eligible 
for high school for the first 
time and stayed in CPS, 
dropped out, or graduated 
over the next four years 


Sample is not affected by the creation of 
APCs or possible increases in the number 
of eighth-grade graduates who drop out 
before enrolling in high school. Excludes 
retained students so that the same students 
are not counted in multiple years. 


Cohort Course Outcomes, 
Graduates Only 


Four-year high school 
graduates only 


Sub-sample shows how CPS performs with 
students who make it through the system in 
four years. These analyses show what 
happens to the "best" students. 


One and Two Year 
Outcomes 


Students who were eligible 
for high school for the first 
time 


Sample is not affected by the creation of 
APCs or possible increases in the number of 
eighth-grade graduates who drop out before 
enrolling in high school. Excludes retained 
students so that the same students are not 
counted in multiple years. 


School Level Outcomes 


Students who entered high 
school in the 1999-00 school 
year 


Uses only one school year, as the clear- 
est comparisons are made across a single 
school year because of variation in the 
number of schools that exist in any given year. 



1 1n 1996, the first year of the new promotion policy for the eighth graders, about 1,800 students were retained in eighth grade 
for poor performance on the Iowa Tests of Basic Skills. This was about three times the retention rate of the previous year. As a 
result, about 1 ,200 students who would have been part of the ninth-grade cohort in 1 996 were excluded. We would expect the 
academic outcomes for the 1 996 cohort to be better simply because the students with poor academic performance did not 
become part of the cohort, although our adjusted trends account for changes in incoming student composition. Furthermore, 
since 1 996, each new cohort has had a slightly larger percent of students held back from entering high school than the previous 
one. In the first three years of the policy, increasing numbers of students were retained in eighth grade or enrolled in APCs rather 
than promoted to high schools.Therefore, each incoming ninth-grade class experienced more holding back of low achieving 
students than the previous one. A subset of those retained students never entered high school (i.e., were never included in a 
ninth grade cohort) either because they dropped out or left the system. The remaining students eventually became part of a 
cohort, some entering one year later, others two or three years later. While the number of eighth-grade retentions stopped 
growing after the first three years of the policy (1 996, 1 997, and 1 998), sixth-grade retention began to affect subsequent ninth- 
grade classes. The effects of third-grade retentions will be seen later, as will the recent increases in first- and second-grade 
retentions. Each incoming ninth-grade class will have had more low achieving students removed from its cohort than in the 
previous year, and this pattern will continue for many years. 
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ITBS 

Controlling for students’ prior achievement in our analyses allows us to estimate what high schools add 
to their students education. Without such a control, schools that attract high performing eighth graders 
appear more effective because their students start out ahead. Likewise, schools that attract low perform- 
ing students appear worse because their students start the ninth grade behind. Results for the system are 
similarly affected. 

A simple solution would be to control for students’ eighth-grade scores on the Iowa Tests of Basic 
Skills (ITBS). However, a single year of test score data is not a precise measure of true ability — students 
may experience a bad test day, or simply miss one or two questions due to carelessness that significantly 
effect their score. In addition, because the Chicago Public Schools uses eighth-grade ITBS scores as the 
basis upon which students are promoted to high school or not, these scores probably do not reflect true 
student ability. As students are required to pass a threshold, those near the threshold are motivated to 
perform at a higher level in order to avoid summer school and those who are well beyond the threshold 
have no special motivation to score well on the test. As a result, some scores might underestimate stu- 
dents’ true ability, others might overestimate them and, the error in the estimates would not be random. 

To more accurately represent the students’ true achievement levels at the end of the eighth grade, we 
ran a two-level HLM using all of the test score data for each student for every year they were enrolled in 
CPS. From this we produced a measure of the latent eighth-grade ability. We included only those eighth 
graders whose test scores were officially included by the school system, and only included previous test 
score data if it was officially included in school system figures. We ran this analysis on cohorts from 1992 
to 2000 simultaneously using test data from 1987 to 2000. The cohort variable indicates the year the 
student was in eighth grade. 

Using students’ test scores as the outcome, the first level of the HLM is the test level. At Level 1, we 
included predictors for the student’s grade (centered around sixth grade), the student’s grade squared 
(centered around sixth grade), a dummy variable indicating whether or not this is a repeated year for the 
student in grades one through seven, and a separate dummy variable indicating whether or not this 
student repeated eighth grade. Thus, the equation at Level 1 was: 

ITBS score = E 0 + E, (Grade) + E 2 (Grade squared) + E 3 (Repeat) +E 4 (Repeat8) + r 

At Level 2, the student level, we modeled the intercept and each slope with dummy variables for each 
cohort year. In addition, because students coming out of bilingual education programs often have a faster 
learning curve than others, we included a dummy variable indicating whether the student had been in a 
bilingual program. The 1992 cohort was the omitted category. The equations at Level 2 were as follows: 

Go = Y 0 + Y, (Bilingual) + y 2 (1993 Cohort) + y 3 (1994 Cohort) + y 4 (1995 Cohort) + y ? (1996 Cohort) + 
y 6 (1997 Cohort) + y 7 (1998 Cohort) + y g (1999 Cohort) + y 9 (2000 Cohort) + u Q 



O 
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g = y o + y, (Bilingual) + y 2 (1993 Cohort) + y 3 (1994 Cohort) + y 4 (1995 Cohort) + y s (1996 Cohort) + 
y 6 '(1997 Cohort) + y ? (1998 Cohort) + y g (1999 Cohort) + Y 9 (2000 Cohort) + u, 

6 2 = y o + y, (Bilingual) + y 2 (1993 Cohort) + y 3 (1994 Cohort) + y 4 (1995 Cohort) + y 5 (1996 Cohort) + 
y 6 (1997 Cohort) + y ? (1998 Cohort) + y g (1999 Cohort) + Y 9 (2000 Cohort) + u 2 

g 3 = y o + y 2 (1993 Cohort) + y 3 (1994 Cohort) + y 4 (1995 Cohort) + y 5 (1996 Cohort) + y 6 (1997 Cohort) + 
y 7 (1998 Cohort) + y g (1999 Cohort) + Y 9 (2000 Cohort) 



g 4 = y o + y 2 (1993 Cohort) + y,(1994 Cohort) + Y 4 (1995 Cohort) + y 5 (1996 Cohort) + y 6 (1997 Cohort) + 
y 7 (1998 Cohort) + y g (1999 Cohort) + y 9 (2000 Cohort) 



O 

ERLC 



Separate models were run for reading and mathematics. We used the empirical Bayes estimates for the 
intercept, the grade slope, and the grade squared slope from the residual files to create a measure of latent 
student ability. Ideally, we would have liked to center our grade and grade squared variables around 



eighth grade so that the 
intercept would have: 

Latent score = (fitted 
value, intercept + 

EB residual, intercept) + 
2(fitted value, grade6+ 

EB residual, grade6) + 

4 (fitted value, grade6sq+ 
EB residual, grade6sq) [+ 
(fitted value, repeat8) if 
student repeated eighth grade] 

TAP 

To not penalize schools 
that work to prevent low 
achieving students from 
dropping out and encour- 
age attendance on test 
day, we imputed test 
scores for students who 
were enrolled in the fall 
but did not have a spring 
TAP score. In this way, 
schools that do not make 
the above efforts and 
have fewer low achieving 
students taking the TAP 
do not have any unfair 
advantage. 



HLM Results for Latent Ability Models 



Intercept 


Reading 


Math 


Intercept 


Reading 


Math 


Intercept 


0.1823*** 


0.5454*** 


Repeat 






Bilingual 


-0.3401*** 


-0.0929*** 


Intercept 


0.0364 


0.1116*** 


1993 Cohort 


-0.1110*** 


-0.0994*** 


1993 Cohort 


0.0095 


0.1031** 


1994 Cohort 


-0.1060*** 


-0.1184*** 


1 994 Cohort 


0.1221** 


0.1242*** 


1995 Cohort 


-0.1249*** 


-0.0118 


1995 Cohort 


0.0820* 


0.1394*** 


1996 Cohort 


-0.0669*** 


0.0210* 


1996 Cohort 


0.1005** 


0.2182*** 


1997 Cohort 


0.0498*** 


0.1410*** 


1997 Cohort 


0.2261*** 


0.3216*** 


1998 Cohort 


0.0605*** 


0.1501*** 


1998 Cohort 


0.1897*** 


0.3022*** 


1999 Cohort 


0.1097*** 


0.1914*** 


1999 Cohort 


0.2126*** 


0.3698*** 


2000 Cohort 


0.2023*** 


0.2366*** 


2000 Cohort 


0.1785*** 


0.3584*** 


Grade, centered 












on 6th 






Repeat 8 






Intercept 


0.2438*** 


0.4748*** 


Intercept 


0.0828 


0.0131 


Bilingual 


0.0292*** 


0.0313*** 


1993 Cohort 


0.0013 


0.1986** 


1993 Cohort 


0.2226*** 


0.0529*** 


1994 Cohort 


-0.1067 


0.1142 


1994 Cohort 


0.2452*** 


0.1006*** 


1995 Cohort 


0.085 


0.0847 


1995 Cohort 


0.2522*** 


0.0737*** 


1996 Cohort 


-0.0393 


0.0844 


1996 Cohort 


0.2902*** 


0.0939*** 


1997 Cohort 


0.1437* 


0.1247* 


1997 Cohort 


0.2861*** 


0.0765*** 


1998 Cohort 


0.2025*** 


0.2803*** 


1998 Cohort 


0.2946*** 


0.1028*** 


1999 Cohort 


0.1956*** 


0.0857 


1999 Cohort 


0.3001*** 


0.1238*** 


2000 Cohort 


0.1514* 


0.1944*** 


2000 Cohort 


0.3014*** 


0.1131*** 








Grade Squared 












Intercept 


0.1581*** 


-0.0534*** 








Bilingual 


-0.0013** 


-0.0081*** 








1993 Cohort 


-0.1387*** 


0.0300*** 








1 994 Cohort 


-0.1443*** 


0.0413*** 








1995 Cohort 


-0.1555*** 


0.0078** 








1996 Cohort 


-0.1519*** 


0.0262*** 








1997 Cohort 


-0.1689*** 


0.0219*** 








1 998 Cohort 


-0.1715*** 


0.0265*** 








1999 Cohort 


-0.1614*** 


0.0378*** 








2000 Cohort 


-0.1584*** 


0.0391*** 









*** 

** 

* 



p < .001 

pc. 01 
p < .05 
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To impute TAP scores, we used students’ gender, age, SES, race, and latent reading and math ITBS 
scores for the entire sample. Using a latent variable regression in HMLM, we were able to use the data 
from both students missing TAP scores and those with TAP scores. However, as we were unable to model 
the school effects at Level 2 and produce a latent variable regression at Level 2 in the current version of 
HMLM, we centered all variables around their school means outside of the HMLM program. 

In this analysis, the Level 1 data files are arranged with one record per variable. All observed data are 
copied into a single variable, outcome. This ‘ outcome” is linked to a set of indicataors that identify 
which specific variable this is. At Level 1, all indicator variables are entered into the equation to predict 
the “outcome” along with a variance component. There is no intercept at Level 1. 



Y = Ifi X + r 

p p 



Where Y is the outcome” and are the indicator variables, p ranging from 1 to 17 in this analysis (see 
the table HMLM Results for Estimating TAP Scores”). If data are missing on some “outcome,” those 
records are just absent. 

The Level 2 equations are simply the coefficients associated with the indicator variables predicted by 
the intercept. These coefficients represent the latent values for each student on each variable (whether 
actually observed or not). 



B = 7 

p 1 



po 



HMLM Results for Estimating TAP Scores 



Latent Variable Regression 
on TAP Score 


Reading 


Math 


Intercept 


-0.0404** 


-0.0588** 


Student SES 


0.0868*** 


0.0066 


Old for grade 


-0.3769*** 


-0.3234*** 


Very old for grade 


-0.2551 


-0.1536 


Male 


-0.0876** 


0.1992*** 


White 


0.3638*** 


0.5155*** 


Asian 


-0.109 


04751 *** 


Latino 


0.1163* 


0.1550*** 


8 th grade reading ITBS score 


1.1430*** 


0.0408 


7th grade reading ITBS score 


1.1224*** 


0.2019*** 


8 th grade mathematics ITBS score 


0.201 7*** 


1.3653*** 


7th grade mathematics ITBS score 


0.3129*** 


1.1976*** 


8 th grade reading score missing 


0.5183 


-0.455 


8 th grade mathematics score missing 


-0.7725 


-0.1539 


7th grade reading score missing 


-0.5953 


0.2487 


7th grade mathematics score missing 


0.7297 


-0.1451 


Student SES missing 


-0.0016 


-0.1291 



** p < .001 
■* p < .01 

* p < .05 



In a latent variable regression, any Level 1 
parameter, , can become either the outcome 
or a predictor in a structured model. We use 
this method to make one of our indicator 
variables the outcome and the others the 
predictors. The latent variable output then 
gives us fixed effects for the predictors on 
whichever outcome we choose. The results 
from the reading and mathematics HMLM 
analyses are shown below. (For a further discus- 
sion of the general HMLM methodology, see 
Raudenbush and Bryk (2002).) 

The coefficients from the table at the left 
were multiplied by the original school-mean- 
centered data and summed across variables. To 
this was added the original school means of 
those variables to produce the imputed scores 
for those missing TAP data. 

Overall, our imputed scores may be higher 
than expected for students with missing scores 
because the model assumes that the student 



missing a TAP score is the same as a student 
with the same SES, race, and ITBS scores who took the TAP. However, if the student is missing a TAP 
score because he has dropped out, it is more likely that he is lower performing than the student who is 
still in school. In this sense, our model produces conservatively estimated imputed scores. 




63 



Student Performance 



59 



School-Level Adjusted TAP Scores 

To predict the high school adjusted mean TAP 
scores, we ran a two-level HLM using TAP 
scores as the outcome, including imputed TAP 
scores for those students missing scores. At 
Level 1, the student level, we used gender, race, 
age, SES, and the latent eighth-grade ability in 
reading and mathematics as predictors. We also 
included a dummy variable to indicate if the 
student attended a summer bridge program 
after eighth grade. In addition, we included 
dummy variables for students entering high 
school from private elementary schools, from 
public schools outside the CPS system, or who 
were once in the system but left and are now 
reentering. At Level 2, the school level, we 
included predictors for the high schools mean 
SES, percent of special education students in 
the school, and percent mobility. We also 
included indicators of the racial composition of 
the student body and a mean of the latent 
reading and mathematics ability of the entering 
ninth-grade class of students. All of the Level 2 
predictors were uncentered. The equations for 
these analyses appear below: 



HLM Results for Estimating the 
Adjusted School TAP Score Means 



Intercept 


Reading 


Math 


Intercept 


9 . 6569 *** 


9.9291*** 


High school SES 


-0.1473 


-0.2705 


Percent special education 


0.0847 


-0.3234*** 


Percent mobility 


1.3726 


2.225 


Predominantly Latino 


0.0489 


0.1413 


Integrated 


-0.036 


0.1423 


Racially mixed 


-0.1158 


0.011 


Predominantly minority 


0.0121 


0.3632* 


Mean latent achievement 


1.2002*** 


1.4201*** 


Male 


-0.0367 


0.1907*** 


White 


0.6006*** 


0J244*** 


Asian 


0.1073 


0.6285*** 


Latino 


0.3016*** 


0.2480*** 


Attended private elementary school 


1.0174*** 


0.4188*** 


Left CPS and returning to the system 


0.0036 


-0.1774** 


From public elementary school 






outside the system 


0.2413** 


-0.0286 


Old for grade 


-0.3086*** 


-0.3203*** 


Very old for grade 


-0.6822*** 


-0.5460*** 


Student SES 


0.0643** 


-0.0066 


Latent 8th grade reading ability 


2.3726*** 


-0.0703* 


Latent 8th grade mathematics 






ability 


0.2010*** 


2.8402*** 


Attended Summer Bridge 


-0.0968* 


0.0747* 



Level 1 

TAP score = fi Q + fi^Male) + fi 2 (White) + 
fi 3 (Asian) + fi 4 (Latino) + fi 5 (Private) + 
fi 6 (From CPS) + fi 7 (Outside) + 

(Old for Grade) + fi 9 (Very Old for Grade) + 
fi 10 (SES) + fi n (Latent Reading) + 
fi 12 (Latent Math) + fi 13 (Bridge) + r 

Level 2 

fi Q = y o + Yi (High School SES) + y 2 (Percent Special 
Education) + y 3 (Percent Mobility) + 
Y 4 (Predominately Latino) + y 5 (Integrated) + 
Y 6 (Racially Mixed) + Y 7 (Predominately Minority) + 
Y 8 (Mean Latent Achievement) + u Q 



** pc. 001 
■* pc. 01 
* pc. 05 



fi . = Y, 

% = y 2 
% = y 3 

% = y 5 

& 6 =y 6 
% = y 7 

= Yg 
fi 9 = Y 9 
= Y,o 
8 ..=Y„ 
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in APCs, and students classified as special education 
make up the bulk of students not taking high school 
level English. 



What Percent of Students Attempt and What Percent of Students Attempt and What Percent of Students Attempt and 

Pass a Social Science Course? Pass a Science Course? Pass a Foreign Language Course? 

One year after elementary school One year after elementary school One year after elementary school 
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Although the percent of students taking the TAP did not change from 1993 to 2000, the composition of 
students who took the test did. Considerably fewer students with very low eighth-grade ITBS reading 
scores took the TAP in the 1999-00 school year compared to 1993-94. 



Fewer Students Scoring Low on ITBS 
Reading Take the TAP the Following Year 





1993-94 


1 994-95 


1995-96 


1996-97 


1997-98 


1 998-99 


1999-00 


Change 


Lowest 

Quartile 


6,173 


6,542 


6,054 


4,573 


4,248 


3,511 


2,982 


Down 

3,191 


Second 

Quartile 


7,289 


6,703 


6,332 


7,196 


7,646 


7,386 


7,779 


Up 490 


Third 

Quartile 


3,742 


3,235 


3,292 


3,803 


3743 


4,618 


4,676 


Up 934 


Highest 

Quartile 


1,591 


1,658 


1,562 


1,898 


1,544 


2,342 


1,991 


Up 400 
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